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Abstract 



For a new family-nonuniversal gauge interaction to be accessible at the LHC it will 
most likely couple preferentially to the third family. By coupling to all members of the 
r~| , third family the production of a new gauge boson (the X with Mx ~ 1 TeV) will lead to 

final states with a distinctive t~^t~ pair. We study the mass reconstruction of the X and 
the cuts that can enhance signal to background. The X boson should be associated with 
. the physics of flavor and in the simplest picture a fourth family. We discuss how the mass 

mixing between the third and fourth families affects the X couplings and a possible mixing 
with the Z. 



1 Introduction 
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^ ■ The LHC is expected to shed light on the origin of electroweak symmetry breaking. There is 
O ' less expectation with regard to the flavor puzzle and the origin of the quark and lepton mass 
spectrum. This is because the latter is usually encoded in a set of Yukawa couplings whose 
^ i origins are typically expected to lie far beyond the scale of electroweak symmetry breaking. In 
this sense the belief in the existence of an elementary scalar field, the Higgs boson, drastically 
reduces our hopes for new understanding about flavor. 

The picture is very different in the absence of elementary scalar fields. In this case the 
physics of flavor will likely have to be understood in terms of the dynamics of gauge theoriesll] 
After all, nature provides an example of the dynamical breaking of chiral symmetries and the 
generation of mass with QCD. If we take the hint then we should consider the possibility of 
gauged flavor symmetries which are dynamically broken. In addition to the broken electroweak 
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"'^ There is also interest in the dynamics of extra dimensions, but it appears that these models require the same 
sort of parameterization of flavor and a fundamental lack of predictiveness as the original set of Yukawa couplings. 
This may be a feature of any attempt at a weak coupling description of flavor. 
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gauge symmetries, we have in mind more badly broken gauge symmetries that connect different 
famihes to each other. These interactions can serve to feed down mass from the heaviest fermions 
to the hghter ones. The physics of flavor will be characterized by scales that range from a TeV up 
to at least several hundred TeV, and these scales will be in some inverse relation to the masses of 
the ferm,o.s that are generated by the respective mteracfcri Besides the small masses that the 
light quark and leptons receive, other effects of new flavor physics on the lightest fermions are 
highly suppressed. Many such effects are characterized by effective 4-fermion operators, but in 
the case of the three light neutrinos their masses may have to arise from 6-fermion operators. The 
existence of neutrino masses in the sub-eV range then turns out to be an independent indication 
of flavor physics below 1000 Tevlfl 

The lightest of the new interactions couples to the heaviest fermions and in this case the 
mass scale of the interaction and the fermion may be similar. In addition the heaviest fermion 
masses should be expected to serve as the primary order parameters for electroweak symmetry 
breaking!^ The most trivial way to accommodate this union of flavor and EWSB is to extend 
the known flavor structure of the standard model by adding a sequential fourth family, which 
will contain these heaviest fermions. We suppose then that the last remaining remnant of the 
flavor gauge symmetry exists down to roughly a TeV. We assume that this flavor interaction is 
strong and that it plays some role in electroweak symmetry breaking. There may be no need for 
a new unbroken gauge symmetry underlying EWSB, such as technicolor. Such a picture would 
constitute the most economical joining of flavor and EWSB physics, and thus it is interesting to 
consider its experimental implications. 

Clearly, the lightest of the flavor gauge bosons should couple to the fourth family. We will 
focus on such a gauge boson associated with a diagonal generator of some original flavor gauge 
group, and call it the X boson [1]. Of most interest to us is the likely coupling of the X boson 
to the third family as well. The main reason for this coupling is that this is the simplest way 
to cancel the gauge anomalies involving the X and the electroweak gauge bosons, by having 
the X charges of the third and fourth families being equal and opposite. This is another link 
between four families and the X boson. With such couplings it is nevertheless still possible that 
the complete flavor dynamics results in only one of the two heavy families receiving the main 
electroweak symmetry breaking mass. The top quark mass must reflect the presence of SU{2)fi 
breaking physics at a higher scale, and a particular 4-fermion operator can feed mass from the 
b' to the t while reducing the impact on the T parameter [21 [3] . 

We have thus sketched a particular justification for the existence of a new massive gauge 

^The form of this inverse relationship may be affected by anomalous scaling. 

•^When a Higgs is present the neutrino mass operator has dimension 5, in contrast to dimension 9 here. This 
explains the vast difference in the suggested scales of flavor physics at which these operators must originate. 
''Any heavier fermions would have to be electroweak singlets, such as the right-handed neutrinos. 
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boson coupling to the third family|§ Our assumption of the simplest possible flavor structure, 
along with the simplest cancellation of gauge anomalies, implies that the X boson couples with 
equal strength gx to all members of the third family. The fact that it couples to both the b quark 
and the r will lead to our search strategy. Thus our analysis will apply to any other scenario 
having a new massive gauge boson with this basic property. We stress that the observation of 
such a gauge boson, nearly decoupled from the two lightest families, would open up a new window 
onto the physics of flavor. Although there has been much attention focussed on signals of new 
massive gauge bosons (often referred to as Z"s) at the LHC, these studies usually involve family 
universal couplings. Our interest is instead with nontrivial flavor physics that is connected with 
origin of the sequential family structure of nature, and for this we need to see family nonuniversal 
physics. 

To calculate the width of the X boson we assume for now that it decays by pair production to 
all members of the third family, and that it is below the threshold for pair production of fourth 
family fermions. The fourth family quarks can be expected to have masses roughly of order 600 
GeVjj Under this assumption the X width is 

Mx 



500 GeV 



60 GeV. (1) 



The quantity we choose to fix in our study is the ratio gx/Mx = 1/(700 GeV), since this basically 
determines the size of possible low energy effects of the X (e.g. see Section 3). With gxjMx 
fixed we shall consider the values for Mx in the range from 700 to 1300 GeV. The corresponding 
couplings are quite strong (these are couplings renormalized at the scale M^) and the X width 
can become quite large, ranging from 84 to 540 GeV. 



2 Signal and Background 

X is produced in collisions due to its coupling to b quarks. Due to the bottom sea quark 
component of the protons, the main parton-level processes in order of importance at the LHC 
are the following. 

hb ^ X 
g(b or 6) Xg{b or b) 

gg - Xbb 

q{b or b) —>■ Xq{b or b) {q = light quark) 

^ There are also small couplings to lighter families arising from CKM related mass mixing effects. Such flavor 
changing couplings are probably less constrained if they occur more in the up sector. 

^The fourth family neutrino and charged lepton may be somewhat lighter. For constraints on the fourth family 
masses see [H [3] . 
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The first process contributes about 2/3 of tlie cross section and tlie second about 1/4. We focus 
on the decay mode X t^t^ . 

The reconstrution of the X mass can be accomplished [Sj even though the r decays produce 
missing energy. The point is that the r's are highly boosted and so the missing component 
of their decay products are close to coUinear with the visible components|l| We can write the 
t"^ momenta as p+/a;+ and p„/a;_, where a;+ and a;_ are the fractions carried by the visible 
components p+ and p_. Then the measured total transverse missing momentum is 

PT={^- 1)PT+ + - 1)PT- (2) 

This vector relation determines x+ and x^. Then the X invariant mass is obtained by scaling 
up the invariant mass determined by the four- vectors p+ and p_ by a factor of 1/ 

For background event generation we use the Alpgen[8]-Pythia[7j combination, while for signal 
generation we use MadEvent[9j-Pythia to accommodate the new physics model. We adopt the 
Pythia tune D6T (due to R. Field) which is based on the CTEQ6L1 PDF§ Since we do not go 
beyond tree-level matrix elements in our event generation, it is important to choose a sensible 
renormalization/factorization scale that can be used consistently for both signal and background. 
We choose to set this scale to the partonic quantity \/§/2 (which goes over to the canonical 
choice of for ti production close to threshold). We note that other popular choices for the 
renormalization scale involve some combination of p^'s and masses of the particles produced, but 
in our case the mass of the X is not well defined when it is a very broad resonance. 

MadEvent is run for the combined set of processes listed above with a constraint that the jets, 
including 6-jets, are required to have pt > 20 GeV. This is due to the issue of double-counting; 
if the bottom sea quarks in the proton are thought to arise from gluon splitting g bb, then 
the first process above could be thought to have the same topology as a diagram contributing 
to the third process. The overlap is expected to occur for 6-jets at small px and large 77, and 
hence the pr > 20 GeV cut. With this cut the third process in total contributes less than 
5% of the cross section and thus any remaining double-counting cannot be significant. We find 
cross sections for X production (in collisions of 7 TeV protons) of 4.4, 2.8, 1.9, 1.4, 1.1 pb for 
Mx = 700, 850, 1000, 1150, 1300 GeV (with gx = Mx/(700 GeV)). 

Alpgen generates background events with MLM jet matching involving 0, 1, and 2 extra hard 
jet samples, with jets having pt > 50 GeV and pseudorapidity \ri\ < 2. PGS4[10] is used for 
detector simulation, with the parameter choice for ATLAS as supplied by the MadEvent package. 

For event selection we use the following cuts, where by 'lepton' we mean an isolated electron 
or muon or a r-tagged jet. The r-tagged jets are in fact essential for the sig nalS 

^The accuracy of this approximation is more than adequate, due to the large natural width of the X. 
^Pythia is run with r decays treated internally and with tt'^'s treated as stable. 

restriction to 1-prong r-tags is also worth considering, which could give comparable results. 
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• at least one pair of oppositely charged leptons, each with pr > 60 GeV, with invariant 
mass > 300 GeV 

• missing energy > 60 GeV 

• Ht> 700 GeV 

• not more than one non-6-tag jet with pt > 60 GeV 

• no jets or leptons with |?7| > 2 and px > 60 GeV 

Very effective is the 300 GeV dilepton invariant mass constraint, which essentially eliminates 
the contribution from the {Z )+jets background. To reconstruct the X mass we consider 

all such pairs of leptons and require that the x+ and x_ obtained by solving ([2]) satisfy < x± < 1. 
Also effective is the requirement of not more than one non-6-tag jet to suppress backgrounds 
involving more jets, which includes tt+jets. A low rate for jets to fake r's, along with the missing 
energy constraint, will help to suppress the pure QCD jet background. The missing energy 
constraint could be increased further if necessary. We will take the r fake rate conservatively to 
be 1% (lower values are described in [6] for ATLAS). 

We are left with tt+jets and ly+jets as the main backgrounds. In our generation of events for 
the former we take both top quarks to decay semileptonically, and for the VT+jets background 
we take the W to decay leptonically. For the VT+jets background, at least one of the two leptons 
used to reconstruct the X will be a jet that fakes a lepton (most likely a fake r). The PGS4 
r fake rate appears to be too large, and to implement our own fake rate we proceed as follows. 
We consider another set of events which has an additional jet which is neither 6-tagged nor 
r-tagged which we use along with an identified lepton in the event to reconstruct the X mass. 
By combining this set of events with the original set we then have both PGS4 r-tagged (fake) 
and non-tagged jets contributing to the reconstruction]^ and to this result we can multiply by 
our estimate of a r fake rate (1%). For the signal and the tt+jets background there are two real 
leptons in the events and in these cases we don't make any correction for the PGS4 r fake rate. 

It is also useful to consider the relative orientation of the three vectors pt+, Pt- and "pr 
appearing in ([2]). In particular consider the vector that bisects the angle between pt+ and pt-- 
Let 01 be the angle between this vector and fx- Then the signal favors | sin(0i)| 1 more so 
than the background. This is due to the fact that pt+ and pt- tend to be close to back-to-back, 
in both signal and background, while for the signal the missing momentum from each r decay 
aligns with pt+ and pr- respectively. Thus a cut |sin(0i)| > 2/3 for example will suppress 
the background more than the signal, especially for larger Mx- We shall impose this cut, while 
noting that other related cuts can also be devised. 

^°The contribution from the non-tagged set is divided by 2 since the charge of the fake will not be correct half 
of the time. 
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Figure 1: The signal for {gx, Mx[GeV]): (1, 700, blue), (1.21, 850, red), (0.5, 700, green). The 
numbers of events are normalized to 1 fb~^. 
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Figure 2: The signal for {gx, Mx[GeV]): (1.43, 1000, blue), (1.64, 1150, red), (1.86, 1300, green). 
The vertical scale differs from the previous figure. 

The X has a production rate that falls rapidly with its mass, and a width that is proportional 
to g\Mx- Both of these effects cause its detectability at the LHC to fall with increasing mass 
even while gx/Mx is held fixed. We show the X mass reconstruction in Figs. (1,2) for Mx = 
700,850, 1000, 1150, 1300 GeV for gx = Mx/(700 GeV). In Fig. (1) we also display the case of 
a smaller couphng, gx = 0.5 for Mx = 700 GeV. By comparing these results one sees how both 
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the height and width of the peak provides information on gx- Fig. (3) shows that our cuts have 
been effective at reducing the backgrounds to levels small compared to the signal. Even a high 
mass broad signal peak can be differentiated from the background since the background peaks 
around 700 GeV. 




500 1000 GeV 1500 2000 

Figure 3: The background for tt+jets (blue) and VT+jets (red). These histograms are stacked, 
not overlayed. The vertical scale differs from the previous figures. 

The X boson should be easy to differentiate from a Z' boson with family universal couplings. 
Through its coupling to light quarks a Z' can be found up to significantly higher mass. And a 
Z' with a mass of order a TeV is constrained to have couplings typical of the Z or smaller and 
would thus be much more narrow than we expect for the X boson. To test the hypothesis that 
the X couples equally to all members of the third family, we need to consider other decay modes 
of the X boson. 

Decays to a pair of quarks can produce hbhb^ bbti and titi final states, but the extraction of 
these signals from background appears to be nontrivial and requires more study. The decay to 
Uj-Vt- can lead to bb +^t, where the missing transverse momentum is that of the X. But the 
latter is not expected to be large compared to the X mass, and the result is that the signal is 
swamped by the Z+jets background where Z vV and the jets fake 6-jets. More promising is 
the decay to the fourth neutrino X v-^tV^t if kinematically allowed. Since mass mixing in the 
lepton sector can lead to the decay v^-i (r, yU or e) + W , this can lead to interesting signatures 
involving varying numbers of leptons and jets. Of course Vt-iVt-i could also be produced directly 
through a Z. 
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3 A fourth family and mixing effects 



We first discuss how mass mixing between the third and fourth families can affect the X couphngs 
to quarks. Let us consider for example the b and b'. In an interaction eigenstate basis with 
B = (b', b) the X couples with coupling strength gx to a vector current 

= '^lI^.QBl + '^Rl.QBn Q=(^l ^-^y (3) 

But the (6', b) fields need not correspond to the mass eigenstates. The unitary transformations 
Bl = UlBl and Br = UrBr to the mass eigenstate basis produce a nondiagonal current 

J, = Bl^^QlBl + Br^^QrBr Ql = UlQUI Qr = UrQUI (4) 

The third family Xbibi and XbRbR couplings are then determined by the respective components 
of the matrices Ql and Qr. These diagonal components vary between —1 and 1, and thus the 
Xbibi and XbR^R couphngs independently vary between —gx and gx- The off-diagonal left- 
handed couplings XbiJi'i and Xb'jJ)L have absolute value < gx and they only vanish when the 
Xbibi couplings are -^gx- The same is true for the right-handed couplings. 

The same story applies to the (T','r) and (z/^,z/^) pairs, with potentially different unitary 

transformations. But we should assume that the X couplings preserve to good approximation 
the usual custodial SU{2) symmetry among t' and b' quarks. This implies that we can write 
the couplings to the third family mass eigenstates in terms of four parameters (fourth family 
couplings are opposite in sign). 

(5) 
(6) 
(7) 
(8) 

If the X boson has some axial quark coupling g^'^ = {g^'^ — g^'^)/2 ^ then it will receive a 
contribution to its mass from the dynamical t' and 6' masses. Given that essentially all of the Z 
mass arises from this source we have the bound 

Ml - Ml' ^ ^ 

An axial coupling can also imply some mass mixing between the X and the Z [Ij. Let us 
denote this by M\-^^ the off-diagonal element of the 2x2 mass-squared matrix. The t' and b' 
contributions cancel in the limit of i!-b' mass degeneracy, but a t loop contributes as long as 



9f 


= 9l' 


Xq 

= 9l 




= 9f 


Xq 

= 9r' 








9^ 


= 9^ 


^9^ 




9'r 


^9l' 
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(7^^ 7^ 0. We can write this contribution to M|jy in terms of ft, which is defined such that ft jv^ 
with V ~ 240 GeV gives the fractional contribution of the t-loop to M; 



I; 



ML = %l'fl (10) 



A standard estimate is /t ~ 60 GeV ^J]. Fourth family leptons could also contribute to this 
mixing, but as we shall see the r' must have mostly vectorial couplings to the X and the Vt-i 
mass is probably not large compared to the t. In terms of the mass mixing M\-^ we have a shift 
of the Z couplings to the third family 

HU^-'^'jUk / = (11) 

The current constraints from 5Y\yjY\y and bY^j'^T are quite strong and imply that substantial 
shifts in the Z couplings are only allowed if the shift in the h coupling is mostly right-handed 
= if big\ = 0.185(7^) and the shift in the r coupling is mostly vectorial [STj. = if 
— 0.866 g^j^). Since the shifts ( ITTl) are proportional to the X couplings, for illustration we 
consider the following choices 

^f = o, gi' = -gr = -9^' = 9x. (12) 

This will essentially give the maximal shifts in quantities other than STb/Tb and STr/Tr- 

Also for illustration we will consider the case where gx/Mx is chosen to produce results 
similar to our Monte Carlo simulations where gx/Mx = 1/(700 GeV). For this we need to set 
gx = 2^gx to make up for the absence of the coupling to The resulting {g'^'^y/Mjr is safely 
compatible with the bound in ([9]). Combining these values with the above results we obtain the 
following shifts in the Z couplings to the third family, 

5gl = 0, 5gl = -5g[ = -bg'^ ^ 0.00386. (13) 

We compare the resulting pattern of deviations to the experimental results [12] in Table 1. 
The experimental ^F^ and bA^ values were obtained from the departures from the standard model 
fit prediction. ^F,- and bA^ were derived from the measured departures from lepton universality, 
assuming Fg = F^ and A(, = A^. bVy^ was attributed to the departure of the measured number 
of neutrinos from three. We see that the X boson induced shifts are compatible with experiment 
except for two of the measurements of the r asymmetry parameter. This apparently rules 
out this picture of maximal Z coupling shifts unless some experimental errors are seriously 
underestimated. The X boson induced shifts can be continuously decreased to zero by increasing 
the size of the vector relative to the right-handed X couplings to quarks. 
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We ignore a possible increase of the width of the X if the decay to h'h etc. became possible. 
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Experimental 


X boson 




0.0022 ±0.003 


0.004 


' 1 ' 


0.0036 ±0.0025 


0.003 




-0.016 ±0.008 


-0.02 


6Ah/At{FB) 
5Ab/Ab(LR) 


-0.046 ±0.016 
-0.013 ±0.022 


-0.009 


bArjAri^B) 

(5A,/A,(LR) 


0.24 ±0.17 
-0.1 ±0.1 
-0.04 ±0.04 


0.27 



Table 1: FB refers to the forward-backward asymmetries, LR the left-right asymmetries from 
SLD, and Vr the tau polarization measurements. 

4 Conclusions 

We have considered a picture for minimal new flavor interactions, where the X, a new massive 
gauge boson, couples preferentially to the quarks and leptons of the third family. The coupling 
can be of order one or more and this can lead to a large width for the X. The X is produced 
at the LHC through its coupling to the h quark, while the decay X — > r+r~ produces the most 
striking signatures. These events will be deficient in non-6-jets. The X mass can be reconstructed 
from the visible decay products of the r's and missing p^. We have discussed event selection 
cuts that quite effectively suppress main backgrounds from tt±jets and W^±jets. This leads to a 
clear discovery potential for the X with mass ranging up to at least a TeV, with only a fb^^ of 
integrated luminosity. 

We have restricted ourselves to X masses at or above 700 GeV because this is roughly the scale 
where we could expect that the physics of flavor comes together with the physics of electroweak 
symmetry breaking. This picture points towards a sequential fourth family with similar mass. A 
fourth family also allows the required cancellation of gauge anomalies associated with a X boson. 
A heavy fourth family has its own signatures at the LHC [3], and in particular t't' production 
provides a signal for which ti production is a dominant irreducible background, just as we have 
found for X production. This indicates that a thorough understanding of the ti sample at the 
LHC will be a prerequisite in the search for new flavor physics. 
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